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Abstract 

We take a unified approach to central limit theorems for a class of 
irreducible urn models with constant replacement matrix. Depending 
on the eigenvalue, we consider appropriate linear combinations of the 
number of balls of different colors. Then under appropriate norming 
the multivariate distribution of the weak limits of these linear com- 
binations is obtained and independence and dependence issues are 
investigated. 
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1 Introduction 



Consider a four-color urn model in which the replacement matrix is actually 
a stochastic matrix R in the manner of Gouet jH]- That is, we start with 
one ball of any color, which is the 0-th trial. Let W„ denote the column 
vector of the number of balls of the four colors upto the n-th trial, where the 
components of W„ are nonnegative real numbers. Then a color is observed by 
random sampling from a multinomial distribution with probabilities (l/(n-f 
1))W„. Depending on the color that is observed, the corresponding row of R 
is added to W[j and this gives W^_|_^. A special case of the main theorem of 
Gouet |HI is that if the stochastic matrix R is irreducible, then (!/(« + 1))W^ 
converges a.s. to the stationary distribution tt of the irreducible stochastic 
matrix R (it should be carefully noted that the multicolor urn model is vastly 
different from the Markov chain evolving according to the transition matrix 
equal to the stochastic matrix R). Suppose the nonprincipal eigenvalues of 
R satisfy Ai < 1/2, A2 = 1/2, A3 > 1/2 respectively, which are assumed to be 
real (and hence lie in (—1, 1)), and Ci, be the corresponding eigenvectors. 
Using n^i = vrR^j = XiH^i it is seen that {l/{n + l))W^^j — > 0. Thus central 
limit theorems are the next interesting statistical results. 

In this article we consider the joint limiting distribution of (X„, Yn, Zn) 
where 

v/n ^n\ogn i(i + 

Special cases of this result are known from Freedman [Sj, Gouet [Zj and 
Smythe ^ and Bai and Hu |T|. Freedman [S], as well as Gouet [7], consider 
two color urn, so that there is only one eigenvector and the corresponding 
nonprincipal eigenvalue can be one of the three types. The identification of 
the norming rates is due to Freedman [H], his technique being the method 
of moments. Smythe jH] considers multicolor urn, but all the nonprincipal 
eigenvalues (or their real parts) are assumed to be < 1/2. Recently Bai 
and Hu [J have considered the case when all the nonprincipal eigenvalues 
(or their real parts) are < 1/2. However to our knowledge, the joint limit 
when eigenvalues of all the three types occur has not been considered. The 
interesting feature of this case, which will be clear from the proof, is the 
differences in the behaviors of the differences of the three components. 
For the above four color set up our main result is 
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Theorem 1.1 (X„,y„, Z„) converges in distribution to {X,Y,Z) where 
X,Y,Z are independent, X and Y are (independent) normals with zero 
means. The convergence of Zn to Z is also in the almost sure sense. 

The variances of X and Y are identified in the proof. However our proof says 
nothing about the distribution of Z, apart from EZ = 0. Some features of 
this Z ma, two color case are discussed in Freedman [5^ . We also need to point 
out the connection of theorem 1.1 with the available results in the literature. 
The available results actually consider norming the vector (W„ — i?W„) and 
not the linear combinations from the eigenvectors. Now the eigenvectors 
Gj^Ij'Cs ^-iid the principal eigenvector u = (1, 1, 1, 1)' span M"^, so that any 
linear combination can be expressed in terms of them. But W^u = n + 1, so 
its effect cancels out after the expectation is subtracted and we are left with 
the linear combinations corresponding to ^1,^2, ^3- The available results in 
the literature divide (W„ — E'W^) by the largest rate, and in the case the 
real part of the nonprincipal eigenvalues is less than or equal to 1/2 (actually 
the rate in that case may be different from ^/n\ogn as will be clear in the 
later sections) derive asymptotic normality, see e.g. Bai et al. [T]. 

We have stated our theorem for the four color model for the sake of nota- 
tional simplicity in the proof. The theorem also extends to situations (with 
more than four colors) where there are more than one eigenvalue(s) of any 
one or more of the three types. These extensions involve the same technique, 
but require more calculations related to the Jordan form of the replacement 
matrix. So we have sketched some of these extensions in separate sections. 
These sections discuss the main theorem in increasing generality along with 
development of suitable notation, and we have indicated the generalizations 
inside these sections. First, all the eigenvalues are considered to be real, 
the Jordan form thus involves only real vectors. Next, the eigenvalues can 
be complex, so the Jordan form involves complex vectors and we deal with 
the real and imaginary parts of these vectors. Another interesting feature 
of these later sections dealing with the Jordan form is the appearance of 
nilpotent and rotation matrices. The final result is given as theorem 5.1. 

The proof of theorem 1 . 1 for the above four color set up is given in the next 
section. It employs an iteration technique involving conditional characteristic 
functions (an example of these iterations occurs in example 2, Pp. 79-80 of 
Basak et al. [3^). We have written this proof in detail, however the proofs for 
the generalizations of the main theorem are only sketched in later sections 
as the ideas are the same. 
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2 Proof of theorem 1.1 

Before starting the proof we collect a few computational details. The column 
vector of the indicator functions of balls of different colors obtained from the 
n + 1-st trial is denoted by Xn+i- It is clear that E{xn+i\^n\ = {^/{n + 
1))W„, where J-'n denotes the cx-field of observations upto the n-th trial. 
This notation leads to 



For the purpose of iteration we shall use a decomposition of the components 
of the Markov chain {Xn+i,Yn+i, Zn+i) illustrated with the first component 
as follows: 



The first term will be expressed in terms of X„ and the second term is the 
martingale difference that will play an important role in our proof in analogy 
with the calculations for the central limit theorem for i.i.d. random variables. 

To write the first term in terms of X„ Z„ respectively) we shall use 
the following approximations 



(2) 



n+l — 




log(n + 1) logn + 1/n + 0{l/n'^) 



1 



1 + (1/n logn) + 0(l/n^ logn) ' 




2n 2n log n 



^^^(l + A3/(J + l)) ~ n 



Using these and the conditional expectation of (j21) it follows that: 




(3) 
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the second of which crucially uses A2 = 1/2. Now let us look at the martingale 
difference terms which are: 



Min+l—Xn+l — E{Xn^l\{Fn} — ^1—^=== ~~r \ / TT^' 

'1 J 1 1' , 1 



Vn + 1 n + 1 V + 1 



A2 / nlogn 



n+1 y (n + 1) log(n + 1) 

X' ^3 

Ms = Zn+l - E{Z n+l\Tn} = A3 " \ ~ T 1 A3 (^) 

It will be seen that the part involving Xn+i^i plays the significant role in the 
second moment calculations. 



2.1 Main idea of the proof 

Now we are ready to start the proof of theorem 1.1. 

Step A: Using (jS)) and the inequality |e*^ — 1| < 2|x| for real number x, 
and remembering that |W^,^j| < cn, so that Xnj ^n^Ynj ^n^Znjvt}''^^ are 
bounded, we can expand (^^^^^^o(\ir?)+it^Y^o{\ir?) ^.^ gg^. 

^|gj(il^n + l+t2V'„ + l+t3Z„ + l) 1^ } 
1 

_g»{*l (1- -SH- )^"+*2 (1- 2„ log „ )yn+f 3^n } ^gi | gi (tl Ml ,„+ 1 +i2 M2,„+ 1 +t3 M3,„+l ) | j 

<2(|ti||X„| + |t2||K„|)0(l/n2) 



1 . . 

< const. ^rjTT, (5) 



for n sufficiently large, say n > uq. 

Step B: Now we want to approximate £'{e*(*i^i."+i+*2*^2,n+i+t3M3,n+i) 

by 



2 -^1 n + 1 2 



+ 1 2 -^2 („ + l)log(n + l) _ (^g^ 

We use the inequality |e*^' — 1 — + < const. \x\^ along with the ob- 
servation that the martingale differences of (jD) are bounded by const, j ^pn., 
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const./ i/nlogn and const. /n^^ respectively (we approximate IIq (1 + A3/ {i + 
1)) ~ n^^). This gives 

^|gj(tlMi,„+l+t2M2,„+l+t3M3,„+l) 1^ j 

-(1 - ]^E[{tlMl^^, + t^M^^+i + tlMl^_,, 

,n+l ,n+l ,n+l 

for n > no- 

To achieve (jH)) a detailed study of the terms of ((Tj) is necessary. We denote 
by the vector whose components are products of the corresponding com- 
ponents of and ^j. Remembering that Xn+i consists of indicator functions 
of observations of balls of different colors, we get 

n + 1 



w ^2 

n + 1 [n + ly J 



n + 1) \og{n + 1) 

, , .2^^±iZl2^1^ _ \2 nlogn 2 
^ '(n + l)log(n+l) '(n + l)3 1og(n + l) " 



i+1 

2^ n+1 ^'^3 ^ A3 
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E{Mi^n+lM2,n+l\Tn) = A1A5 



E(Mi,„+iM3,„+i|J-„ 



,nH-l n 



+ lv/(n + l)log(n + 1) 
w 



V^ + lv/(n + l)log(ra + l) 



— A1A2 
A1A3 



n^logn 



> + 1) Vlog(^ + 1) 
< vr,66 > 



^71 -' 71 f ) 



v/^(ns(i + -^)) 
^ ^ 'v/;^(n^(i + -$T)) 



— A1A3 
A2A3 



n 



A3 

71+1 



n + l(n + l)(l + ^ 



n+l ' 



< 71-, 66 > 



+<! A2A3 



v/(n + l)log(n + l)(n^(l + -^)) 

<5t-^'^26> 



v/(n + l)log(n + l)(n«(l + -^)) 



-AoA 



n logn 



2^31 



A3 

71 + 1 



+ l)log(n + l) (n + l)(l + ^) 



If cr^ is small then we know that |1 — ^ — e "^^^^l < const. a"^. Using this on 
the constant terms of the first two equations of (jHI) we get 

1 _ ^^2>2 < 7r,6' > _ 1^2x2 <7r,6'> 
2 ^ ^ n + 1 2 ^ + l)log(n + 1) 



'1 ^2 <'r.Sl> *2 \2 
' 2 '^l n+l 2 -^2 (. 



n+l) log(n + l) 



< const. - 



n + iy 



(9) 
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Step C: Combining (0), (|7j) and © we get the following basic inequality: 



^|g«{tl^n+l+i2^n+l+t3^n+l) 1^ } 



2^1 n + l 2 '^2 („ + l)log(n+l) 



< const. + (10) 

where we use i?„ to denote the sum of the other constant terms and random 
terms from the right of (jH)) which have not been used in © (this is also 
multiplied by exponentials of imaginary quantities, but those are bounded 
by 1 and will not make any difference). We also use the notation 

— ~7r 1 r~i "o"^2" 



n + l 2 ^ (n + 1) log(ra + 1) 

We then condition again on J^n~i and iterate backwards. While doing so, in 
the exponent the coefficients of ti change as above, we get a sum of C„_j's 
in the exponent, and following iteration of on the right we get a sum of 
conditional expectations of /2„'s and const. Yl^o + l)'^^^- Note that the 
iteration from n + 1 to has changed the coefficient of X„ and F^, and these 
are assumed to be incorporated in C„_i and -R„_i, and so on. Rn-j also 
involves terms like ^(^■n.-i+\+-+c„ ^ '\^^^ seew from steps 1 and 2 that 

these terms are bounded uniformly and will be absorbed in the const, term 
in (jl8|) . We should mention here that the constant term in (0), ((Tj) and Q 
and finally (fTUj) can be taken independently of this iteration because during 
the iteration the coefficients of ti and t2 decrease. 

The main idea of the proof is to iterate the (conditional) characteristic 
function backwards upto a sufficiently large no, and first make n — cx). This 
will make the sum of C„'s independent of no, and the sum of the conditional 
expectations of the -R„'s given jF„p will be bounded by a random variable 
(which depends on the fixed no). Taking expectation of the conditional char- 
acteristic function we get the characteristic function. Then we let no — *■ oo, 
and a further argument gives us the characteristic function. Before we do 
this we provide a few ingredients of the proof in a separate subsection. 
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2.2 Important limits and estimates 

So assume we have iterated backwards upto a sufficiently large uq. For ease 
of exposition we divide the calculations into a few steps. In step 1 we con- 
centrate on the nonrandom terms corresponding to tf and t"^ , which gives the 
form of the characteristic function corresponding to X„ and Yn. In step 2 we 
consider the other nonrandom terms, then in step 3 we handle the random 
(second bracketed) terms. Steps 2 and 3 contribute to the sum of -R„'s. 

Step 1: The calculations here will go into C„. They come from the first 
(nonrandom) terms of the first two equations on the right of (jH)). Because 

-— Ai 

of the presence of the term (1 — ^ ^ ^ ) in the characteristic function, it is 
seen that after iterating backwards upto n^, the (nonrandom part of the) 
coefficient of — (l/2)t^ is 

2^Jn-3+l\——Y- 

no 



where 

In-j+l — ■'-'-1=^+1 



/n-i+1 — n"^ - , i(l - ; Y 



As n — » oo, the above sum clearly goes to 

poo 

A?<7r,e?> / e-(i-2^^)^dx. (11) 
Jo 

Similarly because of the presence of (1— 2„iogn ) characteristic function, 

after iterating backwards upto no, the (nonrandom part of the) coefficient of 



■ 2 ^2 



where 



4^^"-^^^^(, + l)log(, + l) 



= nr=^.+,(i - — L_)2 

22 log 2 



As — > oo, the above sum clearly goes to 

poo 

\l<'n,e2> / e-^dx. (12) 

^0 
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Thus, irrespective of Hq, the (nonrandom part of the) coefficients of — |ti 
and —1^2 SO ^'^ constants as n — oo. At this point note that as we made 
— > cxD the coefficient of X^g in the characteristic function ti ■>/ fn-no+i go^s 
to zero and similarly for the coefficient of F^^, which is tiy/gn-no+i- Thus, 
fixing no, as we let n oo, the characteristic function does not have X„q, Y^^ 
and the nonrandom part of the coefficients of —^tf and — S'^ constants 
independent of uq. This takes care of the sum of C„_j's, j = no, no + 1, ■ ■ ■ , n, 
as we make n — > oo. 

Step 2: The calculations here will go into the upper bound for the sum 
of RnS. The (nonrandom part of the) coefficient of — (l/2)tit2 is 



no 



where 



Clearly 



/^^-...-nr.,,,(i-^)(i-i^). 



I - Ai, 



and combining the y/j + 1 of a/ (j + 1) log(j + 1) with the other y/j + 1, it 
is seen that the term (fT^ is less than 

Vlog(no + 1)^ ^ J + 1 

which goes to 

-====AiA2<7r,eie2> / e-(^^^)^rfx (14) 
v/log(no + 1) Jo 

as n — > cx). Actually here in the expansion of (1 — l/(2i logi))(l — ((1/2) — 
Ai)/i) the important contribution comes from 1 — ((1/2) — Xi)/i, which can 
later be compared with the comments following theorem 5.1. 

The coefficient of — (l/2)tit3 is (we approximate no(l + A3/(/ + 1)) ~ j^^), 



no \ J J 



/n-j+lAlA, 
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where 

- - Xi 

fn-j+i = n"^j_^^(i - ). 

Following the argument of the previous paragraph, as we let n — > oo this 
coefficient is less than 

-^-—X,X,<n,C,^,> e-(5-^^)^da;. (15) 
no Jo 

Similarly a.s n —>■ oo, the (nonrandom part of the) coefficient of —(1/2)^2^3 is 
less than 



V(no + l)log(no + l) 



POO 

A2A3<7r,6e3> / e-"/'c/a;. (16) 
Jo 



Also note that when we iterate backwards the coefficient of Z^g is still ^3 and 
keeping uq fixed as we let n — 00 the (nonrandom part of the) coefficient of 
— goes to 



00 



E^3^. (IT) 

no ' 

Thus, fixing no, the sum of — tit2, —tits, —^2^3, ~^tl, multiplied by their re- 
spective (constant part of the) coefficients, is bounded by a constant as 
we let n 00. The exact form of Fn^ is easily obtained from (fT^ . (fT3j) . (fTBjl 
and (fT7j) . however for us the important observation will be that — > as 
we later make uq ^ 00. 

Step 3: The calculations here will go into the upper bound for the sum 
of -R„'s. We now concentrate on the random terms. First note that 

W 

sup II— — r-vr||, 

no<n<oo Ti -\- L 

where ||.|| denotes the maximum, is a bounded random variable that con- 
verges to a.s. Also Xnl = W^^i/n is bounded by a constant and 
converges to a.s. as Uq 00, hence the same holds for 

sup X^/n. 

no<n<oo 

These two observations show that when we iterate backwards the random 
terms in the coefficient of —t\/2 contribute a random variable less in absolute 
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value than 



const. I sup II— -^-7r||+ sup Xl/n\^ fn-j+i——- (18) 

no<n<oo Tl -r i- nn<n<oo J ' ^ 

no 

The const, term here is an upper bound for e^"~"o~^ '"C" and all the terms 
in step 2 are also to be multiplied by this. Recall that fixing n^ as we 
make n — > oo, the sum fn-j+i-j^ converges to an integral (see (fTTj). so 
that the above sum is bounded by a constant for all n), showing that as we 
make n ^ oo keeping no fixed, the contribution of the random terms to the 
coefficient of —t\/2 is bounded by a bounded random variable. This random 
variable is a constant times the conditional expectation of the random term 
in ()18p given jF„g, which as we can see converges almost surely to as we 
later make no — oo. 

Similarly, for the other terms involving Yn and we use that \J\ogn/nYn 
and Zn/n^~^''^ are bounded random variables. Then exactly as in the previ- 
ous paragraph and following the calculations leading to (jH}, and the other 
coefficients (O, (HH), (fT3|) and (fT^ we see that fixing as we let n — >• oo, 
the contribution of the random terms is bounded by a bounded random vari- 
able, say the conditional expectation given Tn^ of a certain Guq (which goes 
to almost surely as we later make no oo). 

2.3 Completion of proof 

Let us now write = + Gno i that is the remainder term is bounded by 
the sum of a constant and a random term uniformly in n. Notice that 
is actually J-'^o measurable and in the calculations what we really use is its 
conditional expectation given JF„q. Combining steps 1,2 and 3, fixing uq as 
we make n ^ cxd, we get from ()10|) and the previous subsection 

(19) 

with a\ and cr| coming from (jlip and (jl2p respectively. Taking expectation 
and using |i?V^| = |£'i?{V^|jF„Q}| < _E'|i?{V^|jF„g}|, for any integrable random 



limsup |E{e*(*i^"+*2yn+t3^n)|jr^j _ 

oo 

< E{HnQ\J^no} + const. 

no 
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variable V, we get 



^2 ^2 OO 

limsup |£;e^(*i^"+*2^"+*3^") - £;e^*3^"oe-^*?-^*i| < EHr^^ + const. ^ 



no 



j3/2- 
(20) 

Now Zn is a martingale, and in the appendix we show that Z„ is L^-bounded, 
so that Zn converges to some Z a.s. In the calculation so far no is arbitrary. 

We now let uq ^ oo, recalling that the nonrandom F„g converges to and 
that the bounded random variable also converges to almost surely from 
step 3, to get the limiting characteristic function 

This shows that Z is independent of X, F, and that X and Y are independent 
normals. □ 



3 Case of real vectors 



In the previous sections we have considered linear combinations correspond- 
ing to eigenvectors. To consider general vectors we need the Jordan form of 
the irreducible replacement matrix. For simplicity we assume that there are 
only three real eigenvalues. However now there exists a nonsingular matrix 
T such that 

/ 1 \ 

Ai 

A2 



T-iRT = 



v 



where 



/ A, 




1 




1 



A, 



Let us consider the case of Ai. The dimension be di. Then the vectors 
6 = (1, 0, 0, ...)', 6 = (0, 1, 0, ...)', ■■■,U = (0, 0, 1)' transform according 
to the equations Ai^i = Ai^i, Ai^2 = 6 + Ai^2, Ai^a = ^2 + Ai^a, • • • , i.e. 
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in matrix form Ai(6, 6, • • • , ^dj = (Ci. 6, • ■ • , Denoting the ma- 

trix of ^j's for the three matrices Ai,A2,A3 by Si,S2,S3 respectively (and 
necessarily adding O's for the other components) we have 



/ 1 



v 



Ai 



A, 



\ 



A3 / 



(u : Si : S2 : Sa) 



u 



Si : S2 : S3) 



/ 1 



v 



Ai 



A, 



A3 / 



where u denotes the vector (1, 0, • ■ ■ ) of dimension 1 + di + d2 + d^. It may 
be noticed that (u : Si : S2 : S3) is the identity matrix written in a suitable 
form. 

In reality we have to work with not the above matrix of Aj's, but the 
stochastic matrix R. In that case, using the above mentioned Jordan de- 
composition of R, we have to use the vectors T(u : Si : S2 : S3), and the 
equation 



RT(u : Si : S2 : S3) = T(u : Si : S2 : S3) 



/ 1 



V 



Ai 



A, 



A3 / 



As R has principal eigenvalue 1 corresponding to the eigenvector 1 consisting 
of I's, we have Tu = 1. This implies a trivial limit for W'^Tu/ (n+ 1). How- 
ever the limits for the other linear combinations corresponding to W^TSj, i — 
1, 2, 3, are nontrivial and are discussed in the next three subsections. For sim- 
plicity with a slight abuse of notation we shall use the same notation Sj to 
denote TSj. 

Notice that we can write Aj = Aj/j + Fi where Fi is a nilpotcnt matrix. 
The presence of this nilpotent Fi changes our calculations in the previous 
section at certain places and we'll discuss how. We first note that W^_,_iSj = 
W^Sj -|- x'n+i^'^i — W^Sj -|- x'n+i^i-^i (remember the abuse of notation 
mentioned before). We give the most important contributions, the higher 
order terms have been ignored for notational simplicity. 



3.1 Ai < 1/2 

For notational simplicity from now on we shall restrict ourselves to the highest 
order terms significant for the results to hold, and this will be denoted by 
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the notation ~. For A < 1/2, the approximation ■\/n/{n+ 1) ~ (1 — l/(2n)) 
gives 

V""- + 1 V ^ 

leading to the product terms when iterating backwards. On the other hand 
the approximate form leading to the exphcit computations for the conditional 
characteristic function comes from 

1 W , 

= - -^)SiAi. (22) 



As before the most inportant contribution in the conditional covariance comes 
from the first term of the above. Notice that E{xn+iX'n+i\'^n\ consists only 
of diagonal terms and is thus approximately (using Gouet's strong law) D^,, 
meaning the diagonal matrix with components of vr, namely 7ri,7r2, ■ ■ ■, as 
diagonals. This gives for the conditional covariance of ()22|) the approximate 
expression 

^ -a;s;z},SiAi. 



n+l 

This when iterated backwards with terms coming from ()21|) . leads to the 
limiting covariance matrix of the asymptotically normal W,^Si/-\/ri, given 
by 

E -TTn"=.+i(^i - ^^-^)'A;H;D.HiA,niL,+,(/i - ^^-^) 

n->oo ^ — ' 7 + 1 I I 

no 

Jo 

which can be compared with pijl for the case of eigenvector 
3.2 A2 = l/2 



In this case the norming for the central limit theorem is a/ n log^''^"^ n where 
d2 is the dimension of A2. The reason for the 2^2 — 1 power will be clear 
towards the end. First note the approximation 



nlog^^'-'n 2^2-1 



(n + l)log2'^^~^(n + 1) 2n^^ 2nlogn' 
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With this we get 



+ 



E{ , 1^4 

n + l)log2'^2-^(n + 1) 

Vnlog^'^^-in 2n^^ 2nlogn^ 
W' H2A2 



^+ 1 Vnlog^'^^-^n 



i-1 „ 7r) Ino-r) n 



\/n \oi'^'''^n 2nlogn' 



where we have crucially used the form of A2 to cancel the l/(2n)'s occuring 
with opposite signs. This F2 plays an important role in the computations 
later explaining the 2(^2 — 1 power. On the other hand the martingale terms 
for the covariance computations come from 



n + l)log2'^^-^(n + 1) J(n + l)log 



n 



^ - ^)S2A2. (25) 



This gives for the conditional covariance of (j25p the approximate expression 
This when iterated backwards with terms coming from , leads to the lim- 



iting covariance matrix of the asymptotically normal W^Hi/A/nlog^^S^, 
given by 



lim„^^ y n- (12(1-^%^) + ^ 

^(j + l)log2^-i0- + l) -^+1^'^ 2zlogz^ I 



2\/ 



1 p 

A',S'2Z^.H2A2niL,^,(/2(l - -4—) + -^). (26) 



F2 being nilpotent, in the above products only a few terms will be nonzero. 
The consideration of the limits of the nonzero terms will explain the log^''^"^ n 
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term in the norming. We illustrate the case of ^2 = 2 first. The general case 
will follow similarly. 

In this case F2 = 0. Thus the terms are of three types, (i) F2 is omitted 
from both sides (ii) F2 occurs on the left but not on the right and conversely 
(iii)F2 occurs on both sides. Keeping the matrices in order, we bring the 
constant terms together for the purpose of taking limit. 

(i) In this case the products from both sides together reduces to 



Combining this with the l/(j ' + 1) log ^ (j + 1) outside leads to cancellation 
of log^'^^~^(j + 1), leaving us with the sum 



which goes to as n — > 00. 

(ii) F2 occurs on the left but not on the right. The index from which F2 
is taken be ki which lies between j + 1 and n. Then the constant terms from 
the products reduce to 



1 

exp{-(2(i2 - 1) V T-^ :} ~ exp{-(2ci2 - l)(loglogn - loglog(j + 1))}. 

^dogz 





2d2 - 1 
2 




X 




~ — exp{-(2(i2 - 1) (log log n - loglog(j + 1))}. 



Hence combining with the terms outside the sum reduces to 





1 



{logn(logn - log no) - -(log^n - log^no)}. 




n 
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which goes to as n — > oo. 

(iii) In this case we need one ki from the left and one ^2 from the right 
corresponding to the indices of F2^s from the left and the right. From the 
previous calculations the final computation reduces to 



1 " 1 " 1 " 1 

no 7+1 1+1 



no 



1 " 1 



log^'^^"' ] + 

° no 
1 

^ 3 

as ^ oo. 

Thus in the case d2 = 2, only the terms corresponding to the highest 
power of F2 from both sides survives. Exactly the same thing happens for 
general d2- For example when d2 = 3 the following product corresponds to 
the highest power of F2 namely 

2t\ogt ^k, + 1 '^-1+1^' 2nog/ ^k2 + 1 '^^+1^' 2mlogm^' 

and the analysis proceeds as before. The limiting covariance matrix can ob- 
tained from (j2ni), however our main focus is the independence issue. Similar 
calculations can be done using exponentiation technique which is applied in 
Section HI 



3.3 A3 > 1/2 

We expect to get an L^-bounded martingale sequence. Notice first that 
E{Wn+i'^3\J-'n} = W^S3(/3 + j^A-^) . Hcuce the martingale sequence we 
work with is 

z„ = w;s3{^^l(/3 + -^A3)}-^ = w'^E,A-\ (27) 

The following calculation is similar to the calculation in the appendix and 
we have used some approximations for notational convenience. Z„ satisfies 
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the following equation 

n + 1 

~ -r^Z„A3 + xUiS3A;%, (28) 

noting that A„, and A3 commute. To prove L^-boundedness consider 
E E{7in+i^'n+i\^n} ■ Usiug the martingale property and the above decompo- 
sition it follows that 

E{Ln+lZ^'n+l\^n] ~ Z„Z^ - — — Z^AgAgZ^ 

[n-\- ly 

+£;{xUi53A;^A3A'3(A;i)'S'3Xn+i|^„} 

(n + ly 

+Tr{H3A;^A3A'3(A;^)'S^£;{x„+ixUil-^n}},(29) 

where /3 denotes the minimum eigenvalue of A3A3 and we have used properties 
of the trace of a matrix. Approximating £'{£'{xn+iXn+il-^n}} by D^^, further 
expectation of the above inequality gives 

^z„+iz;+i < ^z„z;(i - ^^^-^y ) 

+ccmst.Tr{E^A-'K^K'^{K-')'%D,}- (30) 

We need to find the order of the last matrix so that the above equation 
can be iterated as in the one dimensional case of the appendix, giving L^- 
boundedness of Z„. Essentially we need to show that the terms of A~^ are 
0{n~^'^ log'^^"^ n). We do this for ^3 = 3, the calculations can be generalised. 
First we get a formula for A„. In the following the exponential approximation 
to (1 + l/(j + 1)) has been used for even small values of j for notational 
simplicity, the exact analysis is similar. First we have 

A„ = n-(/3 + ^A3) 

= ni(/3(i + j^) + j^i^3). (31) 
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Using commutativity, the fact that — 0, and 

An can be written as 
ci(n) comes from 



^A3 



" A 1 A 

kx=l Jill 



n 



A3 log fci A3 (log n-log fci ) 



^ gA3logn 



~ n^Mogn. (32) 
A similar computation yields 



fel=l A;2=fei+1 J I i I 



X- 



n n 

fcl = l ^ fc2=fcl + l ^ 

n ^ 



fci=i 

~ n^Mog^n. (33) 

Prom the above two estimations one can now get the following form of 

A;;^ = n-^^(/3 - Ci(n)F2 + d2{n)Fi), 

where d2{n) is easily obtained by multiplying the formulas for A„ and A~^ 
and demanding that the product be I^. This shows that the terms of 
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A~-^(A,^^)' are 0(?2~^^^ log^ n). With this one can go back to (|3(J|) to prove 
L^-boundedness. Similar calculations can be done using exponentiation tech- 
nique which is applied in Section HJ 

Then the analysis of section 2 proceeds to show independence of the weak 
limits (strong limit for Z„). We may state the analogue of theorem 1.1 as 
follows: 

Theorem 3.1 In the case all eigenvalues are real, we consider the linear 
combinations corresponding to the vectors as identified at the beginning of 
section 3. The weak limits of the normalized linear combinations correspond- 
ing to eigenvalues A<l/2, A = l/2 and A > 1/2 are independent. 

For the different eigenvalues all of which are less than 1/2, there may be de- 
pendence among the weak limits coming from the Jordan blocks for different 
eigenvalues (see theorem 5.1 later). For real A = 1/2 there is only one Jordan 
block (the situation for complex A with real part 1/2 is somewhat different). 
For A > 1/2 the weak limits coming from the Jordan blocks corresponding 
to different A's are correlated, one instance of this has been computed in the 
appendix. Thus they are not independent in general. 

4 Complex eigenvalues 

For complex eigenvalues we consider another canonical form which is simi- 
lar to the Jordan canonical form. This form comes from considering the real 
vectors coming from the real and imaginary parts of the complex vectors cor- 
responding to the complex Jordan form. Special cases of this decomposition 
has been studied in Smythe j^. We first consider three types of eigenvalues, 
one of each type as before (i.e. with real part less than 1/2. equal to 1/2, 
and greater than 1/2). There exists a nonsingular matrix S such that 



/ 1 



S^RS 



A, 



cl 



A, 



v 



Ae3 / 
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where 



Ar 



I 



\ 

I 

B, ) 



and 



B, 



-A 



/ is a 2- dimensional identity matrix and rest of the elements are 0. Let the 
dimension of Acj be Idd. As before we partition the matrix SI (this I has 
dimension 1 + 1[dc\ + dc2 + c^cs)) into a vector of ones and Sj, % = 1,2,3 
with number of columns in Si equal to 2dci. These vectors give us the linear 
combinations. 

Notice that, here we can write 



A, 



where Id is an identity matrix of dimension 2dci, Cd is a block diagonal 
matrix of same dimension as A^j. Each block, say Di, is of dimension 2, 
where 

1 
-1 



Di 



Fci is a nilpotent matrix of order dd, i.e., F^-"' = and dd is the least such 
integer. 

First observe that the rotation matrix Di satisfies Df = —I, Df = —Di, Df 
/,■■■, where I is the identity matrix of same dimension as Di. Also, it is to 
be noted that the matrices Id, Cd and Fd commute with each other. Thus, 



fcl /ci +fc2 Cci +fc3-F'ci 



= e'^I4cosih)Ic^ + smih)C,,][Y,ikiFl^)/J^■]■ (34) 

i=i 

We will mention briefly how the proof of theorem 11.11 go for the complex 
roots with the presence of the nilpotent matrix and the rotation matrix. We 
note that W'^_^-^Si = W'^Si + x'n+i^Si = W^Si + x'n+iSi^d- We give the 
most important contributions, the higher order terms have been ignored for 
notational simplicity. 
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4.1 Ai^ < 1/2 

In this case, since ^Jn/{n + 1) ~ (1 — l/(2n)), it is to be noted that, 

^{^^1^4 . }^(,, _ IhlZ^y (35) 

Now iterating backwards we get the product terms as before. Thus, 

^ - :J^)^iAci. (36) 



As before the most important contribution in the conditional covariance 
comes from the first term of the above. Notice that E{xn+iX'n+i\'^n\ con- 
sists only of diagonal terms and is thus approximately (using Gouet's strong 
law) Dj^. This gives for the conditional covariance of ()3fjj) the approximate 
expression 

-K'^iS'iDt^SiKci- 



n + l 

This when iterated backwards with terms coming from ()35|1 . leads to the 
limiting covariance matrix of the asymptotically normal W^S'i/i/n, given 
by 

no 

A:^iS;D^SiAein^=,+i(/ci - 1^2i^^)| 

/ e-(5^^i-^-i)'^A;iSiD^5iA,ie-(^^^i-^-i)«rfs^ (37) 
Jo 

which can be compared with for the case of eigenvector ^1. From the 
calculation in (j34j) . it can be seen that 

dci-l 



e ..... — = e-(5-"-)^/,i[cos(sAi,)/,i + sin(sAie)ai][^(sF,i)Vj!] 



-(|7ci-Aci)s 

which is an integrable function, and hence (j37j) is finite. 
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4.2 A2r = l/2 

In this case the norming for the central hmit theorem is \/ n log^'^''^"^ n where 
2dc2 is the dimension of Kic- From the calculation of the covariance matrix 
the reason for the 2dc2 — 1 power of the the logarithm will be clear. The 
approximation 



nlog^'^^^-^n ^^^^ 242 



leads to 



(n + l)log^'^=2"^(n+ 1) 1n'^ 2nlogn- 



n+ l)log^'^=^~^(n + 1) 
W;52 1 242-1 ^ 



+ 



Vndog^^^^^ 2n 2nlogn 
+ 1 ^n\oi^^^-^n 



n 



= ^|p=^(«l-|*pi) + ^a. + ^k), (38) 

log^'^^^-^n 2nlogn n n 

where the form of is used to cancel the l/(2n)'s occuring with opposite 
signs. We later discuss the role of Cc2 and Fc2 in the computations that 
explains the power of the logarithm. Notice that the computation of the 
covariance matrix depends on the martingale terms 



n + 1) log2'^^^-^(n + 1) ^(n + 1) \og^'^^^-\n + 1 

1 W 

(xUi - -^)^2Ac2. (39) 



Thus, the approximate expression for the conditional covariance of (j39p is 
found as ^ 

(n + l)log^^--(n + l)^^^^^^^^^'^"- 
Iterating backwards with terms coming from (j38j) leads to the limiting co- 
variance matrix of the asymptotically normal a/ n log^'^''^"^ n, given 
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by 



tr(j + i)V'^'-'(i+i) 



xn'L,^.fe(i-^i^) + ^c., + %}. (40) 

zzlogz z z J 

We shall now use exponentiation to simplify the calculations (the same could 
be done in section 3.2). Observing that, 

rrn (r /-, 24^^x ^ac^ J_ ^^^^ v 



— g Z^i=j + 1 2ilogi -'c2^Z^i=j + i i ^c2-t-Z^i^j + i i 

^ log isiffu /c2+A2cC.2 log(j$x)+^=2 log(^) 



e 2 logu+ij 

e-^'°^^/c2 [/c2C0s(A2cl0g(-^)) +Ce2Sin(A2cl0g(-^))] 
dc2— 1 

[5](F,2log(-^))V^!]- (41) 

k=l 
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Combining the contribution of the term in (|41|) to the two sides of (j4(Jj) we 

get 



(j + 1) log'^^'-^j + 1) ^-^+1^-^^^- 2zlogz ' ■ t 



nr=,-,i(/c2(i - -:^) + — c^c2 + — 



77T-j -) H -Oc2 H — , 

2z log % I % 



1 _(2rf,2-l)logj,|p^ 



(j + l)log2'^--^(j + l) 
X [/c2 cos(A2c log(— — )) + sin(A2c log(— — ))] 

J + 1 J + 1 



n 



X [ 5^ (F.'2 l0g(-— ))VA:!]A;252^.52A.2 

X [/c2 cos(A2c log(— -)) + Cc2 sin(A2c log(— -))] 

J + 1 J + 1 

dc2-l 



x[J](F,2log(-— ))VA:!] 



= , . , ,M 2d,2-i [^^2 cos(A2c log(-— -)) + sin(A2c log(-— -))] 
(j + 1) log^'''^' ' n J + 1 J + 1 

[E(^'^2log(-^))VA:!] 
fc=i 

Ac2'S'2^7r5'2Ac2 [/c2 COs(A2c log(— ^y) ) + Cc2 sin(A2c log(— ^y) )] 

[^(F,2log(^))VA:!] (42) 

Now arguing as in section |21 and observing that terms involving sine and 
cosine are all bounded, one finds that except the coefficient of the high- 
est power term of Fc2 i.e. F^2^~^, the coefficients of other terms go to zero 
when n (yo. Observe that highest power terms of Fc2 would be multiplied 
by cos2(A2.1og(-^)) (i.e. (1/2)[1 + cos(2A2.1og(-^))]), sin2(A2e log(-f^)) i.e. 
(l/2)[l-cos(2A2clog(-^))]), or, terms such as, sin(A2c log(-fY)) cos(A2clog(-^)) 
(i.e. (l/2)[sin(2A2clog(-T^))]), separately. Thus, the highest power terms of 
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Fc2 with sine function give the coefficient 



1 >A 1 r(logn-log(j + l))2'^-2-2 



log'-^^^-^n ^ j + 1 I ((42-1)!) 



sin(2A2c(log^ - log(j + 1))) ) 
2 / 

.logn-logno ^2d,2-2 sin(2A2cM) 



log^'^--^nyo ((42-1)!)^ 
(logn — logno)^'^^^^^^ 



0( '^,"Z7' ) -0 (43) 



(seen by integration by parts) as n — >• oo. Similarly, with cosine function, it 
gives 

1 ^ 1 r(logn-log(j + l))2'^=2-2 



W'^^-^n^^ j + ll ((42-1)!) 

cos(2A2c(logn - log(j + 1))) ) 
2 J 

logn-logno 2dc2-2 



^2d,2-2 cOs(2A2cU) 



log2'^--inyo ((42-1)!)^ 2 

/ (logn -logr^o)^^-^-^ ^^ 

( , 2d.2-i ) ^0 as n ^ oo. (44) 



l0g-'^c2-l^ 

Now, the terms that involve multiplying by the 1/2 only, give 

1 ^ 1 (logr2-log(j + l))2"'-2-2i 



log'-^^^-^n ^ j + 1 ((42-1)!) 

J="0 

;>logn-logno ^2dc2-2 



log2'^--inyo ((42 -1)!)^ 2 

1 / (logn — logno)^'^''^"^ 



2(242-1) ((42-1)!)^ V log''^^-^ 
1 



n 



2(242-1) ((42-1)!)^ 



(45) 



as n ^ oo. Thus the limiting covariance matrix can obtained from ()4Up . 
Notice that it does not involve A2c- 
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4.3 Asr > 1/2 

Here also, we show that W^^S^A'^ is an L^-bounded martingale sequence, 
where A„ = n^^(7e3+-^Ac3)- Notice first that E{W'^^,S-i\Tn} = W;,S3(/c3+ 
^^Aca). Hence the martingale Z„ satisfies the following equation 

Z„+i-Z„ = W'^Ss{{Ics + ^Ac3r'-Ic3)A-' + x'n+iSsAcsA-l, 

n + 1 

^ " ■ • ■ — 1 I O A -1 A A A -1 



-Z„A„Ac3A„ + 

77/ ~|~ -L 

~ ^Z„A,3 + x;+i53A-%3, (46) 

n + 1 

since A„, A~^ and Acs commute. To prove L^-boundedness, first observe 
-E{Z„+iZ„+i|^„} ~ Z„Z„ - -pYy^Z^AcsA^gZ^ 

+E{xUl'53A;^A,3A;3(A;;l)'5^Xn+l|^n} 

+Tr{SsA-'AMK'yS'sE{xn+iXn+i\^n}}m 

where Pc3 denotes the minimum eigenvalue of AcsA^g. Approximating 
E{E{x 

n+iXn+il-^ ^}} by Dtt, further expectation of the above inequality gives 

£;z„+iz;+i < ^z„z;(i - ■^-^^) 

+const.Tr{SsA~'A,sA'^{A-'yS'M. (48) 

We now find the order of the last matrix so that the above equation can be it- 
erated as in the one dimensional case of the appendix, giving L^-boundedness 
of Z„. We show this by showing that the terms of A~^ are 0(n~'*'3 log*~^ n). 

An = H^(/e3 + -Ae3) 
J 

= H^(7,3(l + ^) + ^Ce3 + ^F,3). (49) 

Using commutativity of /c3, Cc3 and and the fact that F^^^ = 0, A„ can 
be approximated as 

A ~ gA3rlogn/c3+Cc3logn+Fc3logn 
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Hence 



~ e-^^'-i°s"/,3[cos(-A3.1ogn)/,3 + sm(-A3.1ogn)C,3] 
x[£i^c3(-logn)7fc!] 

k=l 

~ n^'^'- log"^^'-^ n. (50) 
Thus, A^^ = 0(n~^^'' log'^''^"^ n) and from (|15j) one gets L^-boundedness of 

Then the analysis of section 2 proceeds to show independence of the weak 
hmits (with strong hmit for Z„). We may state the analogue of theorem 1.1 
as follows: 

Theorem 4.1 In the case eigenvalues are complex, we consider the linear 
combinations corresponding to the vectors as identified at the beginning of sec- 
tion 4- The weak limits of the normalized linear combinations corresponding 
to eigenvalues Re{\) < l/2,Re{\) = 1/2 and Re{\) > 1/2 are independent. 

For the different eigenvalues all of which have real parts less than 1/2, there 
may be dependence among the weak limits coming from the (modified) Jor- 
dan blocks for different eigenvalues (see theorem 5.1 later). For Re{X) = 1/2, 
there may be different (modified) Jordan blocks corresponding to different 
Im{X). However, inside Re{X) = 1/2, the weak limits coming from (modi- 
fied) Jordan blocks of different dimensions are not independent, in general. 
For A > 1/2 the weak limits coming from the Jordan blocks corresponding 
to different A's are correlated (similar to the real eigenvalue case computed 
in the appendix). Thus they are not independent in general. 

5 General case 

In the general case we decompose the replacement matrix into a (modified) 
Jordan form as in the previous two sections. That is, corresponding to real 
eigenvalues we take the form as in section 3, and corresponding to complex 
eigenvalues by considering the real and imaginary parts of vectors we take 
the form from section 4. Without loss of generality, now we can consider 
only the real parts of the eigenvalues, and the linear combinations will come 
from the (modified) Jordan form. 
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There are now three types of blocks, for Re{\) < 1/2, for Re{\) = 1/2 
and the last type is for Re{\) > 1/2. According to our previous notation, 
there exists a nonsingular matrix M such that 



/ 1 



M-^RM 



G 



1 



G2 



V 



where 



/A 



A, 



■i,2 



G, 



\ 




and Aj jS are either of the form of Aj as in section (jS)) or as in section Q. 
Also notice that, for each i = 1,2, 3, there is a positive integer < ki < Ui 
such that Aj 1, . . . , Aj blocks correspond to real eigenvalues and the rest of 
the rii — ki blocks correspond to complex eigenvalues. It can be observed 
that k2 < 1, and it is also assumed that the blocks inside G2 which have the 
same dimension (i.e. same ^2 or dc2) are arranged next to one another and 
put into one subblocks. 

Let us recall that the linear combinations come from the columns of MI 
which we write with an abuse of notation as (1 : Mi : M2 : M3). With 
appropriate normalizations they decompose into the following three classes, 
independent in the limit. 

Theorem 5.1 1. Re(A) < 1/2: For the linear combinations corresponding 
to columns of Mi, the normalization is y/n and the limit is normal. The 
covariance is given by with Gi replacing Ad (and Mi replacing Si) and 
we have to use the decomposition ofGi combining the features of the real and 
the complex cases. 

2. Re(A) = 1/2: Recalling the arrangement inside G2, in this case the 
linear combinations correspond to columns of M2. For the subblock of G2 
having dimension 62 or dc2 for the original A2^k's (of the same dimension), 
the normalization for the corresponding columns of M2 is a/ n log^'^"^"^ n ( or 
x/nlog^'^^"^ n as appropriate) and the limit is normal. The limits for dif- 
ferent subblocks are not independent, in general, and for each subblock the 
covariance can be found from \40^ to by decomposing the subblock of G2 
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combining the features of the real and the complex cases (and replacing S2 by 
the column submatrix 0/M2 corresponding to the subblock of 02)- 

3. Re(A) > 1/2: For the linear combinations corresponding to columns of 
M3, W^M^A^^ is an LF'-bounded martingale sequence, where An = nQ~^(/3+ 
j^Gs), and I3 is an identity matrix of the same dimension as G3. The 
covariance among some of the components of the (almost sure) limit may be 
nonzero, even though rates are different, implying dependence. 

To summarize part one and two of the above theorem, observe that 
{l/^/n) is the only normaihzation constant for the part one, i.e., for W'^Mi 
and (not necessarily zero) covariances are obtained between different Jordan 
blocks in this part. Whereas, for part two, let us take M2 = [M2^i : . . . : M2,„2] 
where M2j's correspond to the different Jordan subblocks. Then 



■'n2,l 

Cn2,2 



( Cn1.\ \ 
\ Cn2,ni ) 

is asymptotically normal with covariance matrix given below, where C„2,j is 
a diagonal matrix of dimension p^j with each entry as ( 1 / a/ n log^*^™^"^ n) . 
Here dmj equals to d2j if it corresponds to a real eigenvalue (as in Sectional), 
and it is dc2j if it corresponds to a complex case (as in Section H}, whereas 
Pmj equals to c?2j if it corresponds to a real eigenvalue, and it is 1dc2j if it 
corresponds to a complex case as in Section |^. This is a case for asymptotic 
mixed normality. In this case, the typical entries of the limiting covariance 
matrix of W^M2C„, say V2, can be seen as follows, as in (jlKjl . 



1 

1{dm, + - 1) ((rf™, - \Wml - 1)!)^^"^""2,,-.^ 



^2(j,0 = 7773 7T773 TTT73 —k^jM'^,D^M2,ikmi, 



where A^j is the subblock of G2 corresponding to M2J. 



6 appendix 

Suppose f/„ and Vn are normalized linear combinations corresponding to 
eigenvectors .^3,^4, with eigenvalues A3, A4, respectively both of which are 
real and greater than 1/2. We want to show that the limiting covariance is 
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nonzero. This technique has been used in the proof of lemma 3.1 of Freedman 
0. Un and Vn satisfy the following equations: 



A 



3 



A4 



n+1 

+ j+l) ^ ^ n+l 



K+i-K = A4 --^K. (51) 



Using the martingale property it follows that 

A3 A4 

n+l n+l 



ns(i + ^)n^(i + ^) ^ + 1 

EUn+iV^+i = EUMl 



A3 A4 

n+l n+l 



1 + 1 + 

^ n+l ^ n+l 

A3A4 „ w. 



ns(i + ^)n^(i + n + l 



<E—^,Ua> .{52) 



Notice that by the dominated convergence theorem and Gouet's strong law, 

' n+l 



-^"^ converges to < vr,^3^4 >. Iterating the above equation and using that 



no(l + ^) ~ we get (remember A3, A4 > 1/2) that EUnVn converges to a 
nonzero quantity. In particular the same technique yields the L^-boundedness 
of Z„ of section 1. 
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